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Background: Satisfaction witii services represents a key component of tlie user's perspective, 
and user satisfaction surveys are the most commonly used approach to evaluate the aforemen- 
tioned perspective. The aim of this discursive paper is to provide a critical overview of user 
satisfaction surveys in addiction treatment and harm reduction services, with a particular focus 
on opioid maintenance treatment as a representative case. 

Methods: We carried out a selective critical review and analysis of the literature on user sat- 
isfaction surveys in addiction treatment and harm reduction services. 

Results: Most studies that have reported results of satisfaction surveys have found that the 
great majority of users (virtually all, in many cases) are highly satisfied with the services 
received. However, when these results are compared to the findings of studies that use different 
methodologies to explore the patient's perspective, the results are not as consistent as might be 
expected. It is not uncommon to find that "highly satisfied" patients report significant problems 
when mixed-methods studies are conducted. To understand this apparent contradiction, we 
explored two distinct (though not mutually exclusive) lines of reasoning, one of which concerns 
conceptual aspects and the other, methodological questions. 

Conclusion: User satisfaction surveys, as currently designed and carried out in addiction 
treatment and harm reduction services, do not significantly help to improve service quality. 
Therefore, most of the enthusiasm and naivete with which satisfaction surveys are currently 
performed and interpreted - and rarely acted on in the case of nonoptimal results - should be 
avoided. A truly participatory approach to program evaluation is urgently needed to reshape 
and transform patient satisfaction surveys. 

Keywords: patient satisfaction, substance abuse treatment services, harm reduction services, 
patient-centered evaluation, service user perspective, user involvement 

Introduction 

Interventions and programs addressing drug use and related harms have traditionally 
been evaluated primarily by "hard" indicators or criteria,' including program retention 
rates, use of the primary drug of abuse and other psychoactive substances, morbidity/ 
mortality, and criminal activity, among others. These criteria are relevant and necessary 
to inform health care policies; however, they do not necessarily reflect the priorities 
of those who use harm reduction or addiction treatment services.^-^ Moreover, there 
is evidence showing that user perspectives do not correlate with staff perspectives,'''^ 
a well-described phenomenon known as the "paradox of de synchrony."'' Therefore, it is 
clear that a more inclusive and comprehensive evaluation of the process and outcomes 
of interventions should include users' assessment of these variables.' 
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Satisfaction surveys, a key component of assessing the 
service user's perspective,' have long been the most com- 
monly used method of exploring this perspective.^ User 
satisfaction surveys assessing addiction treatment and harm 
reduction services have consistently found high skewness 
toward positive satisfaction scores.'"'^ While this could be 
interpreted with great enthusiasm, other hypotheses should 
be considered, such as publication bias, lack of clarity of 
norms against which to compare the treatments, the psycho- 
metric properties of the instruments used, and the lack of an 
underlying theoretical framework of patient satisfaction."''' 
The aim of this paper is to provide a critical overview of 
user satisfaction surveys as currently carried out in addiction 
treatment and harm reduction services, with a special focus 
on opioid maintenance treatment (OMT) as a representa- 
tive case. First, we argue the lack, relevance, and need for 
a more patient-centered approach to program evaluation. 
Then, we analyze the main features of client satisfaction 
surveys conducted at OMT centers as well as conceptual and 
methodological issues that may explain why the results of 
such surveys often diverge from results obtained via other 
data collection/generation methods. Finally, based on the 
discussion of some limitations of patient satisfaction surveys 
as currently performed, we suggest alternative directions for 
inquiry and research focused on a truly user-participatory 
approach to program evaluation. 

Towards a more user-centered 
approach to program evaluation 

When conducting a user-centered evaluation of OMT 
services, several indicators or assessment areas should be 
considered: a) user expectations and objectives for the treat- 
ment or intervention,'''"' b) patient beliefs or opinions about 
medication (primarily methadone or buprenorphine),""^' 
c) user appraisal of the relational dynamic with interven- 
tion providers^^"^" and the degree of perceived participation 
in decision-making,^^'^* as well as patient preferences for a 
participatory or deferential role,^'-^* d) indicators of user- 
perceived quality,^' e) client perception of unmet needs,^" " 
and f) level of satisfaction with treatment.^"-^'-'^ 

Despite the relevance and validity of the aforementioned 
factors in facilitating adherence to the intervention'"""^' 
and in measuring service quality," " very few programs 
or centers evaluate these factors on a routine basis. This is 
surprising given that program adherence and service qual- 
ity are two areas in which the importance of these variables 
is readily apparent. On one hand, these variables influence 
program acceptability and given that acceptability can 



affect adherence, these variables may mediate - at least 
partially - the efficacy of the interventions. On the other 
hand, the accreditation process of some types of treatment 
centers (eg, methadone treatment centers) requires the 
periodic evaluation of certain variables (generally, patient 
satisfaction) to assess quality.^** '' 

These variables (or more generally speaking, the user's 
perspective) have generally received scant attention""'"' and 
have been largely ignored by the community of mainstream 
researchers,*^ who do not acknowledge the relevance of such 
variables in designing, providing, and/or evaluating addiction 
treatment and harm reduction services. This lack of research 
on user perspectives contrasts with the many studies evaluat- 
ing interventions and treatments. This is particularly worri- 
some considering that input from service users is essential if 
we are to define, implement, and evaluate these interventions 
and programs, ensuring they meet the needs of the users 
themselves."^ Needless to say, a more in-depth understanding 
of user perspectives of interventions and centers would allow 
us to better interpret the efficacy and quality of interventions, 
as well as to plan and implement measures to improve those 
parameters. Without this knowledge, the risk of imderesti- 
mating or distorting the value of treatment services to users 
through the usual assessment practices is high."" 

In recent years, interest in including users' perspectives to 
evaluate interventions that address substance use appears to 
be growing. This interest has resulted in the increasing use of 
patient-reported outcome measures in both clinical trials and 
usual clinical practice."'""' This emerging framework could 
be related to the current trend toward altering the traditional 
hierarchical relationship between health care professionals 
and patients, ie, a relationship based on the classical model 
of medical practice, which implies a biased and exclusionary 
perception of drug users as noncompetent persons."** "' 

Satisfaction surveys, as stated above, are a key compo- 
nent of assessing the service user's perspective' and are the 
most common and most studied approach to exploring the 
aforementioned perspective.* Nevertheless, it is important 
to keep in mind that most classification systems of the 
opportunities or levels of participation in addiction treat- 
ment and harm reduction services consider user satisfaction 
surveys to be "low-involvement" activities (Table 1).'"~" 
In fact, some authors prefer to exclude user satisfaction 
surveys from the definition of participation because such 
surveys do not entail the existence of mechanisms to 
modify planning processes in response to the survey's 
findings, nor do they imply that service users participate 
as equal interlocutors.'^ 
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Table I Levels of user-participation or opportunities for user-participation in addiction treatment and harm reduction services 



Degree of user involvement 


Type of participation 


Example of activity 


High 


Activities implying a share in decision-making 


User's representatives involved in service planning 
committees 

User's representatives attending staff meetings 
User's representatives involved in staff recruitment 
User's representatives involved in staff performance 
appraisal 


Mild 


Activities in which service users have nondecision- 


Users involved in writing or reviewing informative 




making roles 


fact sheets or educational materials 
Users involved in staff training 




Activities promoting and supporting user 


Availability of an adequate space and schedule 




involvement 


within the service to run users own support groups 
Availability of publications or information about the 
activities of drug user organizations 


Low 


Activities related to providing information to 


User councils 




or receiving information from service users 


User forums 
User satisfaction surveys 
Suggestion box 
Complaint process 

Adequate display of information concerning changes 
on policies or service hours 




User-participation built into the values and 


Charter of rights 




policies of the service 





Notes: These levels represent a range of opportunities for user-participation. In practice, and due to very diverse reasons (eg, availability of the activity by the service or 
user's awareness and motivation), participation does not occur in all of these areas, in all services, or in the case of all users. Adapted with permission from Bryant J, Saxton M, 
Madden A, Bath N, Robinson S. Consumer participation in the planning and delivery of drug treatment services: the current arrangements. Drug Alcohol Rev. 2008:27(2); 1 30— 1 37. 
Copyright © 2009, John Wiley & Sons.^' 



Assessment of client 
satisfaction in OMT 

The number of validated self-reports specifically designed 
to assess satisfaction with addiction treatment^''-^^ and, more 
specifically, with treatment for opioid dependence,^'''" are 
quite limited. Client satisfaction with OMT has been primar- 
ily assessed through a) specific questions,^^-^' b) nonvalidated 
questionnaires designed ad hoc,'"" ''' c) nonvalidated adapta- 
tions of instruments originally designed to assess satisfaction 
with mental health services or other types of health care 
services,''- or d) validated generic scales designed to assess 
satisfaction with any type of health care center (ie, not spe- 
cific to addictions)." ''^"''' 

Patient satisfaction research conducted with instruments 
described above shows that most OMT clients (virtually 
all, in many cases) are highly satisfied with the care 
received. "■'''•'^•''''•''' However, these findings clearly contradict 
a) specific comments made by particular OMT patients on 
many occasions and situations;'"'' b) most studies that have 
used a qualitative methodology to explore the views and 
experiences of OMT clients, the results of which are more 
nuanced, or in some cases, reflect a clearly ambivalent attitude 
toward OMT;''''"''' and c) the results obtained by satisfaction 
surveys that include (in addition to items or a scale to evaluate 
satisfaction) either a list of potential problems or negative 



treatment-related aspects from which the respondent must 
select, or open-ended questions requiring a short response 
about which aspect(s) of the user's experience with the center 
he/she finds least satisfying. ^'■'*'^'' Interestingly, these latter 
mixed-method studies have allowed researchers to identify 
significant and/or frequent problems reported by clients 
who, concurrently, showed a high level of satisfaction with 
OMT 

To explain the reasons behind this apparent discrepancy, 
two distinct but nonexclusive arguments can be made: one 
line of argument is more closely related to conceptual aspects, 
while the other is mostly concerned with various method- 
ological questions. 

Conceptual confusion between patient 
perceptions and patient satisfaction 

There is a conceptual confusion originating from the 
interchangeable use of the terms "patient satisfaction" and 
"patient perceptions." Satisfaction refers to the patient's 
perception of the extent to which his/her expectations, needs, 
or desires have been met. Satisfaction is thus, a particular 
kind of perception, but it is not the only one." This confiision 
has increased as most scales that assess satisfaction with 
addiction treatment have been developed without an under- 
lying theoretical framework. Although various theoretical 
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models of the "treatment satisfaction" construct exist, it 
seems to be assumed implicitly - and naively - that this is 
a construct with a universal meaning, or one that does not 
require greater clarification.'^ In fact, some authors believe 
that the concept of treatment satisfaction seems to have 
been imposed on service users, with little consideration of 
its relevance.'^ This situation is even more complex in the 
case of OMT, in which meanings linked to the construct of 
"patient satisfaction" and their respective operationalization 
can lead to even more confusion and ambiguity. For instance, 
from both a clinical and research perspective, it seems neces- 
sary to distinguish, at a minimum, between a) holding dose 
(ie, the dose that prevents subjective and objective opioid 
withdrawal symptoms over the 24-hour interdosing interval); 
b) dose adequacy (ie, the dose that allows patients not to use 
heroin, not to experience withdrawal symptoms nor heroin 
craving, not to show symptoms of overmedication, and in 
the case of heroin use, to hardly experience any subjective 
effects); c) satisfaction with medication; and d) satisfaction 
with treatment. 

Even when patients report a high degree of satisfaction 
with treatment, this does not necessarily entail that they have 
a similarly positive perception of the treatment received. 
High satisfaction scores can be due to negative initial 
expectations^'"" or may reflect, among other nonexclusive 
alternatives, beliefs of the type "they do the best they can"''-'* 
or "really, that is not their job",'* or service user preference to 
demonstrate a positive interaction with the people who care 
for them (social etiquette)," or the limitations derived from 
the user's dependent position in the health care system.'' 
Given the consistently positive results from patient satisfac- 
tion surveys, it is no wonder that many authors believe these 
types of survey are little more than rhetorical practices that 
provide us with the comforting illusion that we are listening 
to patients,*" or that these surveys are mechanisms by which 
managers and clinicians seek to legitimize and maintain the 
status quo of treatment centers.'''-*'"*' 

Methodological and procedural issues 

The second line of argument concerns diverse methodologi- 
cal and procedural issues related to satisfaction surveys. 

Unidimensional versus multidimensional approaches 

Patient satisfaction instruments can be classified according 
to factor structure as either unidimensional (ie, those that 
report a single overall satisfaction factor) or multidimensional 
(ie, comprised of several factors corresponding to differen- 
tiated facets or dimensions of treatment satisfaction).*'' *' 



Dissatisfaction rates are, in general, higher in surveys that 
use multidimensional scales compared to those obtained 
with surveys that have used global or unidimensional 
instruments,*' *'' although this comparison is not completely 
appropriate due to methodological differences arising from 
the type of instrument used. 

Affective versus factual focus 

Patient satisfaction scales can be divided into two types 
depending on the contents: factual measures (ie, objec- 
tive reports) and affective measures (ie, subjective rating 
scales).*' *' Factual instruments, which are designed to avoid 
value judgments, focus on the service user's experiences. 
These types of instruments contain specific questions 
about those experiences in order to acquire objective data 
("How often do you go to the center to take or pick up 
methadone?"; "How long after the scheduled appointment 
time do you usually have to wait to be seen?") or to verify 
whether certain processes and specific events have occurred 
or not ("Have the side effects of methadone been explained 
to you?"). In contrast, affective instruments focus on the 
patient's opinion about those experiences which reflects 
his/her preferences and expectations ("Do you think the 
frequency of your visits to the center to take/pick up 
methadone is excessive?"; "Are you satisfied with the wait- 
ing time between the scheduled appointment time and the 
actual start of the appointment?"; "Are you satisfied with 
the information you have received about the side effects 
of methadone?"). 

There is a lack of validated factual instruments among the 
satisfaction scales available for use in addiction treatment and 
harm reduction services. However, when such instruments 
do become available, it seems likely that the experience 
observed in other health care settings - ie, affective measures 
typically detect higher levels of satisfaction with treatment 
than factual instruments*' ** - will be repeated. 

Affective self-report instruments, particularly those 
with a unidimensional factor structure, are unlikely to 
provide information that will be useful to improve health 
care.*' *'''*' This is due to the way that most questions are 
formulated in these instruments; in many cases, it is not 
clear what elements, aspects, or processes need to be modi- 
fied when a client's response indicates dissatisfaction,'"" 
and it is even less clear how such changes should be 
effectuated. However, the use of multidimensional instru- 
ments (especially the factual measures) can remediate, at 
least partially, this weakness, as has occurred in other areas 
of health care.** '^-'' 
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Early drop-outs and refusals to participate 

We must not neglect a discussion of two important variables 
that have received insufficient attention in many of the patient 
satisfaction studies carried out to date in the addictions 
field: a) the number of early drop-outs from the program 
being evaluated^ '■^^■'''•'^ and b) the nonresponse rate to the 
survey.^''"'"" Early drop-outs and no response could be 
reflecting low levels of satisfaction with treatment. These 
factors contribute to the uncertainty of the results obtained 
and are a threat to both the internal validity and generaliz- 
ability of such results. 

A variant of the Hawthorne effect 

The Hawthorne effect is a positive and transitory change in 
a behavior being evaluated under experimental conditions. 
The change in behavior occurs not because of any altera- 
tions in the independent variable, but rather because 
participants know they are being observed or studied. 
By analogy, levels of satisfaction with treatment could, 
in some cases, reflect or be sensitive to this effect. Sitzia 
and Wood"" indicate that both the extra attention implicit 
in the process of data collection and the apparent interest 
in the user's level of satisfaction could lead to a positive 
perception of the service or center and, consequently, to 
higher scores. 

The phenomenon of response shift 

Finally, the response shift phenomenon'"^ merits discussion 
as a potential additional explanatory factor for the high levels 
of patient satisfaction typically observed. This phenomenon 
has been widely studied in the field of health-related quality 
of life, although it is equally relevant to any field that uses 
self-report instruments.'"^ Villar Lopez et al'"'' highlighted 
the importance of response shift in evaluating satisfaction 
with treatment. The response shift phenomenon in health 
care settings refers to the process of adaptation to changes 
caused by the disease or treatment.'"' More specifically, 
Sprangers and Schwartz'"^ define response shift as a change 
in the meaning of the self-evaluation of the target construct 
as a result of a) a change in the internal standards of measure 
(scale recalibration); b) a change in the values themselves - in 
other words, in the importance of the domains or components 
that make up the target construct (scale reprioritization); or 
c) a redefinition of the target construct (reconceptualization). 
Many years of substance use and a relatively long OMT could 
potentially modify the client's internal standards, values, and/ 
or concept of treatment satisfaction and, thereby, overestimate 
those levels of satisfaction. 



Future directions 

Most of the instruments currently available to assess user 
satisfaction with addiction treatment and harm reduction ser- 
vices are probably unable to adequately measure satisfaction 
due to how those instruments were developed. In general, 
the available instruments are based on a theoretical model 
that is often only vaguely defined and which evaluates areas 
of interest chosen by the researchers, service providers, or 
policy makers.'"^ '"'' It seems evident that self-reports devel- 
oped in this manner are more likely to reflect the priorities of 
the clinicians, researchers, and/or administrators rather than 
those of the service users.'"''"'"'"' This implies that questions 
about aspects of treatment and care that are relevant to 
patients would not be included, even though other variables 
that patients might consider irrelevant are incorporated.^' If 
true, to some extent this represents a threat to basic bioethical 
principles, such as autonomy,"" and to the validity of data 
gathered with these tools,'" questioning the results obtained 
with these instruments. 

Users of addiction treatment and harm reduction 
services - like users of any other health care services and 
programs - have a complex and highly nuanced experience 
that can be more appropriately and fruitfully captured via 
more generic, open-ended questions that are formulated in 
terms of the user's experience (not the satisfaction) with the 
service. Additional approaches include other qualitative tech- 
niques of data gathering, such as participant observation,"^"' 
focus groups," '' or in-depth interviews."''""'' Likewise, the 
critical communicative methodology"'"'" would seem to be 
particularly pertinent and promising in this area of research. 
Although the information provided by these methods can be 
extremely valuable for improving and redesigning interven- 
tions, a periodic quality assessment relying heavily on these 
approaches appears to be of questionable practicality under 
current circumstances. 

The use of the critical communicative methodology or 
qualitative data gathering techniques must be an essential 
requirement for developing the much-needed new scales to 
measure treatment satisfaction'^" or perception of treatment'^' 
in a more patient-centered manner (Table 2). '"' '^^ A genuinely 
patient-centered assessment of satisfaction with treatment 
will not be feasible without patients participating in the devel- 
opment of the instrument (eg, identifying the dimensions 
and variables to be assessed, drafting the items). Moreover, 
the aforementioned techniques should also play a major role 
in helping health professionals and researchers to directly, 
through the patient's own words, come to gain a thorough 
understanding of the patient's perspective (eg, expectations, 
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Table 2 Classification of instruments to measure patient satisfaction according to the degree to which they incorporate the patient's 
own perspective 



Type 



Description 



Patient satisfaction 

scale developed by patients 

Patient-centered satisfaction scale 



Patient satisfaction scale valued 
by patients 



Patient satisfaction scale irrelevant 
to patients 



Note: Data from Trujols et al. 



A patient satisfaction scale (PSS) developed by patients (PSS-DP) is a PSS developed completely 
from the patients' perspective. The main differentiating characteristic of this type of PSS lies in the 
fact that all those involved in the different phases of its development are (or have been) recipients 
of the service to be evaluated. 

A patient-centered PSS (PSS-CP) is a PSS that explicitly incorporates, to a greater or lesser extent, 
the patients' perspective. Even though other stakeholders (eg, clinicians, researchers, family 
members, and/or administrators) have participated in developing the instrument, the patients 
themselves have made major contributions, usually in the initial phases of I) generating the domains 
or variables to be evaluated, 2) writing the questions, and/or 3) evaluating some characteristic of 
the initial or pilot version of the instrument (eg, pertinence or comprehensibility of the items). 
A PSS valued by patients (PSS-VP) is a PSS that, developed v^ithout direct patient participation, 
is considered as such (ie, valued or appreciated) by most of them because it reflects, at least in 
part, their perspectives. To determine whether a PSS developed without patient participation 
can be considered patient-valued, qualitative research methods such as focus groups or cognitive 
interviews must be performed. 

A PSS irrelevant to patients (PSS-IP) is a PSS that has been I) entirely developed without any patient 
participation, and 2) assessed by patients as irrelevant or of no interest or value. A priori, assuming 
no additional information is available, any PSS developed without any direct patient participation 
should be placed, at least provisionally, in this category. 



perceived participation in decision-making). Having a 
deeper understanding of this perspective is an essential part 
of any efforts to improve the quality of addiction treatment 
and harm reduction services. This perspective cannot and 
should not be limited to user satisfaction with the service, 
and neither should user-participation in quality improvement 
efforts be restricted to answering a questionnaire. That said, 
if planners, managers, and workers involved in addiction 
treatment and harm reduction services are unwilling to make 
the necessary effort to achieve an in-depth understanding of 
users' perspective - and to act on any new insights obtained 
in the process - then these methodologies are unlikely to be 
implemented, and thus, authentic patient-centered satisfac- 
tion surveys will not be carried out. 

Given this situation, it is not surprising that some authors, 
both in the drug-use intervention field* as well as in other 
areas of health care'^' pose this question: Do current satis- 
faction surveys form part of an emerging process of shared 
decision-making whose objective is to improve health care 
delivery and clinical care practices or, to the contrary, are 
such surveys a mere token gesture (ie, an empty ritual)'^'' 
from which no changes will emerge? In accordance. Madden 
et aP^ affirm that "satisfaction surveys will be of only limited 
value in opioid treatment settings until there are agreed qual- 
ity standards and formal mechanisms to educate consumers 
about their rights and the standards of care to which they are 
entitled". Without these changes and a commitment to act 
on the knowledge so acquired, it is unlikely that meaningful. 



rather than tokenistic, modalities of user involvement will be 
widely implemented in the near future. In fact, service users 
should not be asked to participate in a satisfaction survey if 
their input will amount to nothing. Moreover, it should be also 
emphasized that the aforementioned changes are unlikely to 
increase the right of service users to participate if their most 
basic needs are not met. 

Patient participation should extend beyond assessing 
and improving the quality of care. The involvement of 
drug/service users in the design and implementation of spe- 
cific programs and interventions, especially in the field of 
harm reduction, has been remarkable, with users, at times, 
displaying a higher level of competence than the public 
health institutions themselves. Similarly, considering 
that users' rights should not be limited to treatment needs 
alone, this dialogical and participatory dynamic should 
be extended to other areas such as addiction research'^" and 
the development of drug policies."' 

Limitations 

The aim of this article was to provide a critical review of a 
comprehensive topic (ie, satisfaction with addiction treat- 
ment and harm reduction services) rather than a systematic 
examination of a focused research question. Therefore, a 
nonsystematic review was judged as the best approach for 
covering a wide range of issues relating to the topic reviewed. 
However, this approach implies not to systematically identify, 
select, appraise, and synthesize all research on the topic. 
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representing a possible limitation of the present study. In any 
case, as several authors have pointed out, traditional narra- 
tive reviews are more appropriate for comprehensive topics 
and systematic reviews are better suited for focused topics.''^ 
Additionally, our attempts to identify relevant studies went 
beyond the efforts usually made in narrative reviews. Spanning 
the last decade, our experience in research on satisfaction with 
addiction and harm reduction seryices^'^ "'^'''^''''^'"'^^ '*" has 
rendered us current on this topic. In this regard, the PubMed 
interface to search MEDLINE has been used. A search strategy 
favoring sensitivity over specificity is periodically performed 
using different combinations of addiction (eg, substance abuse, 
heroin, cocaine) or intervention-specific (eg, methadone, 
detoxification, needle exchange) terms and words related to 
the patient's perspective (eg, satisfaction, perception, view). 
In addition, reference lists included in the selected articles are 
normally examined for other relevant studies. 

Conclusion 

The current approach to user satisfaction surveys does not 
significantly contribute to the improvement of service quality. 
Therefore, most of the enthusiasm and naivete with which user 
satisfaction surveys are currently conducted and interpreted - 
and rarely acted on in cases of nonoptimal results - in addiction 
treatment and harm reduction services should be avoided. A 
truly participatory approach to program evaluation is needed 
urgently to reshape and transform patient satisfaction surveys. 
Similarly, there is a need to implement alternative research and 
quality-improvement initiatives that foster meaningfial, rather 
than merely tokenistic, service user involvement. In turn, this 
will surely improve the relevance as well as the psychopolitical 
validity (both epistemic and transformative)'" "'' of research 
and interventions in the substance use field. 
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